1. Field of the Invention
The present invention relates to a technology for embedding or reading additional information, such as copyright information and editing information, in picture data or the like. The present invention relates to, for example, a digital watermark embedding processor, a digital watermark detecting processor, a digital watermark embedding-processing method, and a digital watermark detecting-processing method that are used to execute a process for embedding in a picture a digital watermark (also called “data hiding”) as additional information that cannot be recognized in normal observation conditions, and a program storage medium and a program used therewith.
2. Description of the Related Art
With advance in digital technology, digital recording/playback devices that eliminate problems caused by repeatedly executing playback processing, such as picture quality deterioration and sound quality deterioration, have come into widespread use. In addition, various types of digital content such as various pictures and pieces of music have become able to be distributed by using media such as digital versatile disks and compact disks or by network.
In digital recording/playback technology, quality similar to that of the original data is maintained because data does not deteriorate even if the data is repeatedly recorded or played back, differently from analog recording and playback. Widespread use of this digital recording/playback technology results in a flood of unauthorized copies, so that it is a big problem from the point of view of copyright protection.
To cope with copyright infringement caused by unauthorized copies of digital content, a system for preventing unauthorized copying has been proposed. The system functions by adding copy control information for controlling copying of digital content, reading the copy control information in mode of recording or playback of content, and executing processing in accordance with the read copy control information.
There are various systems for controlling copying of content. For example, among them, a common one is the Copy Generation Management System (CGMS). When the CGMS is applied to analog video signals (which may be called “CGMS-A”), among 20 bits as additional information to be superimposed on an effective video part in one specified horizontal interval in the vertical blanking period of the brightness signal, for example, on an effective video part in the twentieth horizontal interval in the case of an NTSC (National Television System Committee) signal, two bits are superimposed as copy control information. When the CGMS is applied to digital video signals (which may be called “CGMS-D”), the signals are transmitted in a form in which they include 2-bit copy-control information as additional information to be added to digital video data.
In the CGMS, the 2-bit information (hereinafter referred to as the “CGMS information”) has the following meanings: “00” indicates that content may be copied; “10” indicates that content may be copied once (copying is permitted in only one generation); and “11” indicates that content is prohibited from being copied (strict prohibition of copying).
The above CGMS is one type of common copy control system. In addition, there are other systems for protecting the copyright of content. For example, digital broadcasting by broadcasting stations employs a copy generation control system that, by storing a digital copy control descriptor in program arrangement information (i.e., service information) included in transport stream (TS) packets constituting digital data, performs copy generation control in accordance with the digital copy control descriptor when data received by a receiver is recorded in a recording unit.
Since the above descriptor is added as bit data to, for example, the header of content, it is impossible to completely exclude a possibility of interpolation of the added data. A system that is advantageous in excluding the possibility of data interpolation is digital watermarking. It is impossible to view or perceive a watermark in normal conditions for playing back content (picture data or audio data). Embedding and detection of the watermark can be performed only by executing a particular algorithm or by a particular device. When content is processed by a device such as a receiver or a recording/playback unit, by detecting the watermark, and controlling the processing in accordance with the watermark, reliable control is implemented.
Information that can be embedded by using a watermark includes, not only the above copy control information, but also various types of information such as content copyright information, content modification information, content structure information, content processing information, content editing information, and content-playback-system information. For example, by using watermarks to embed pieces of editing information in content editing mode, each editing step performs recognition of the type of step by referring to its watermark. This editing information is embedded as a new watermark in content in, for example, each step of editing content, and a process such as removal from the content of the watermark is finally performed.
Watermark Embedding
Various types of techniques for embedding and detecting digital watermarks in data have been proposed. Digital watermarking used in this embodiment is a technique based on data as original signals, for example, statistical properties of pictures. Accordingly, digital watermark embedding processing based on the statistical properties of pictures is described below.
When the original picture in which watermarks are to be embedded is represented by P, and a digital watermark pattern to be embedded in the original picture P is represented by W, the digital watermark pattern W satisfies the following expression:Σi,jWi,j=0  (1)
By way of example, the original picture P and the digital watermark pattern W are defined by the following expressions:                               P          =                      (                                                            21                                                  22                                                  23                                                  25                                                  24                                                                              22                                                  24                                                  28                                                  30                                                  26                                                                              21                                                  23                                                  27                                                  31                                                  29                                                                              22                                                  25                                                  30                                                  30                                                  28                                                      )                          ⁢                                  ⁢                  W          =                      (                                                                                                      -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                                                        1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                                              -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                                                        1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                                                                  -                      1                                        ⁢                                                                                                            1                    ⁢                                                                                            )                                              (        2        )            
In expression (1), for brevity of representation, the size of the original picture P is set to 5 by 4 pixels. Adjacent elements in the original picture P are respectively set as close values since adjacent pixels in a picture have, in general, close levels. Although the example shown in expression (2) shows that the original picture P and the digital watermark pattern W are set to have identical sizes, it is not required that the sizes of the original picture P and the digital watermark pattern W be not identical. If their sizes are not identical, arithmetic operations are performed for an area in which the original picture P and the digital watermark pattern W overlap with each other.
The digital watermark embedding processing is executed based on the following expression:M=P+W  (3)where M represents a picture generated by embedding the digital watermark pattern W in the original picture P. For the example shown in expression (2), the value of M is follows:                                                                         M                =                                ⁢                                  P                  +                  W                                                                                                        =                                ⁢                                                      (                                                                                            21                                                                          22                                                                          23                                                                          25                                                                          24                                                                                                                      22                                                                          24                                                                          28                                                                          30                                                                          26                                                                                                                      21                                                                          23                                                                          27                                                                          31                                                                          29                                                                                                                      22                                                                          25                                                                          30                                                                          30                                                                          28                                                                                      )                                    +                                      (                                                                                                                                                      -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                                                                        1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                                                                      -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                                                                        1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                            )                                                                                                                          =                                ⁢                                  (                                                                                    20                                                                    23                                                                    22                                                                    26                                                                    23                                                                                                            23                                                                    23                                                                    29                                                                    29                                                                    27                                                                                                            20                                                                    24                                                                    26                                                                    32                                                                    28                                                                                                            23                                                                    24                                                                    31                                                                    29                                                                    29                                                                              )                                                                                                 (        4        )            Detection of Digital Watermarks
Detection of digital watermarks uses the digital watermark pattern W. Detection of watermarks for the original picture P in which the digital watermark pattern W is not embedded is defined by the following expression:x=P·W  (5)watermark the operator “·” represents the inner product of the matrix, and x represents the inner product of the original picture P and the digital watermark pattern W.
Inner product x is a value in the proximity of zero because the sum of the elements of the digital watermark pattern W is zero (see expression (1)), and adjacent pixels in a picture tend in general to have close values. In the example shown in expression (2), the inner product is calculated by the following expression:                                                                         x                =                                ⁢                                  P                  ·                  W                                                                                                        =                                ⁢                                                      (                                                                                            21                                                                          22                                                                          23                                                                          25                                                                          24                                                                                                                      22                                                                          24                                                                          28                                                                          30                                                                          26                                                                                                                      21                                                                          23                                                                          27                                                                          31                                                                          29                                                                                                                      22                                                                          25                                                                          30                                                                          30                                                                          28                                                                                      )                                    ·                                      (                                                                                                                                                      -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                                                                        1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                                                                      -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                                                                        1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                                                                          -                              1                                                        ⁢                                                                                                                                                            1                            ⁢                                                                                                                                            )                                                                                                                          =                                ⁢                                                      (                                                                  -                        21                                            +                      22                      -                      23                      +                      25                      -                      24                                        )                                    +                                                                                                                         ⁢                                                      (                                                                  +                        22                                            -                      24                      +                      28                      -                      30                      +                      26                                        )                                    +                                                                                                                         ⁢                                                      (                                                                  -                        21                                            +                      23                      -                      27                      +                      31                      -                      29                                        )                                    +                                                                                                                         ⁢                                  (                                                            +                      22                                        -                    25                    +                    30                    -                    30                    +                    28                                    )                                                                                                        =                                ⁢                3                                                                                 (        6        )            
Next, for a picture M having the embedded digital watermark pattern W, similar arithmetic operations are performed. Digital watermark detection for the picture M having the digital watermark pattern W is performed similarly to the above manner by calculating inner product x′ in accordance with the following expression:                                                                                           x                  ′                                =                                ⁢                                  M                  ·                  W                                                                                                        =                                ⁢                                                      (                                          P                      +                      W                                        )                                    ·                  W                                                                                                        =                                ⁢                                                      P                    ·                    W                                    +                                      W                    ·                    W                                                                                                                          =                                ⁢                                  3                  +                                      W                    ·                    W                                                                                                                          =                                ⁢                                  3                  +                                                            (                                                                                                                                                                  -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                        )                                        ·                                          (                                                                                                                                                                  -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                                                                        -                                1                                                            ⁢                                                                                                                                                                        1                              ⁢                                                                                                                                                        )                                                                                                                                              =                                ⁢                                  3                  +                                                                                                                         ⁢                                                      (                                                                  +                        1                                            +                      1                      +                      1                      +                      1                      +                      1                                        )                                    +                                                                                                                         ⁢                                                      (                                                                  +                        1                                            +                      1                      +                      1                      +                      1                      +                      1                                        )                                    +                                                                                                                         ⁢                                                      (                                                                  +                        1                                            +                      1                      +                      1                      +                      1                      +                      1                                        )                                    +                                                                                                                         ⁢                                  (                                                            +                      1                                        +                    1                    +                    1                    +                    1                    +                    1                                    )                                                                                                        =                                ⁢                                  3                  +                  20                                                                                                 (        7        )            
The inner product x′ of the original picture P and the digital watermark pattern W is in the proximity of the inner product of the digital watermark pattern W itself, while the inner product of the original picture P and the digital watermark pattern W is a value in the proximity of zero. In other words, the inner product x′ is a value in the proximity of the value of the following expression:W·W  (8)
The value of the above inner product W·W can be used as a measure of embedding intensity of digital watermarking. In embedding of a digital watermark pattern, when target inner product W·W is large, the embedding intensity of digital watermarking is described as “strong”, while when target inner product W·W is small, the embedding intensity of digital watermarking is described as “weak”.
In addition, when the absolute value of the inner product x of the original picture P and the digital watermark pattern W, and the absolute value of the inner product x′ of the picture M in which digital watermarks are embedded and the digital watermark pattern W are large, the detection intensity of digital watermarking is expressed as “strong”, while when the inner product x and the absolute value of the inner product x′ are small, the detection intensity of digital watermarking is expressed as “weak”.
Strong detection intensity of digital watermarking may be expressed as “great” or “high” correlation between a picture and a digital watermark pattern, while a state in which weak detection intensity of digital watermarking may be expressed as “small” or “low” correlation between a picture and a digital watermark pattern.
For various pictures, by finding the inner product x of the original picture P and the digital watermark pattern W, and the inner product x′ of the picture M in which digital watermarks are embedded and the digital watermark pattern W, relative distributions of the frequency of the inner products are expressed by probability density functions f and f′ as shown in FIG. 1.
Determination of whether or not digital watermarks are embedded needs to use distribution around zero of the values of the inner product x of the original picture P and the digital watermark pattern W, and distribution around the inner product W·W (of the digital watermark pattern W itself) of the values of the inner product x′ of the watermark-embedded picture M and the digital watermark pattern W. By finding the inner product x″ of a picture from which watermarks are to be detected and the digital watermark pattern W, and comparing the found inner product and a threshold value (th), determination of whether watermarks are embedded is performed. Specifically, for the purpose, the following expressions can be used:x″<th then no-watermarkx″≧th then watermarked  (9)
Expressions (9) indicate that, when the inner product x″ of a picture from which watermarks are detected and the digital watermark pattern W is less than the threshold value (th), it is determined that the picture is not digitally watermarked, and when the inner product x″ is not less than the threshold value (th), it is determined that the picture is digitally watermarked. These are expressed as shown in FIG. 2.
The threshold value (th) is determined based on the statistical properties of the probability density function f of inner product x and the probability density function f′ of inner product x′. To determine that an inner product exceeds the threshold value (th) despite the fact that digital watermarks are not embedded, that is, to determine that digital watermarks are embedded despite the fact that the digital watermarks are not embedded is called “false positive”. Conversely, to determine that an inner product does not reach the threshold value (th) despite the fact that the digital watermarks are embedded is called “false negative”.
By setting probability PPP in which “false positive” occurs and probability PFN in which “false negative” occurs, the threshold value (th) is determined. If the central position W·W of the distribution of probability density function f′ is not sufficiently large, it would be impossible to satisfy probabilities PFP and PFN. Accordingly, the process for embedding the digital watermark pattern W in the original picture is changed from expression (3) to the following expression:M=P+cW  (10)where c represents a scalar value that is not negative.
In accordance with the change, the intensity of embedding the watermarks is changed from the inner product W·W of the digital watermark pattern W itself to an inner product multiplied by the scalar value, cW·W.
To determine the threshold value, first, the probabilities PPP and PFN required for an application using watermarks are set, and boundary values thFP and thFN used therefor are set. At this time, it is necessary to satisfy the following expression:thFP≦thFN  (11)
Next, the central position cW·W of probability density function f′ is determined while considering the statistical properties of probability density functions f and f′′so that the above expression thFP≦thFN is satisfied. Finally, the threshold value (th) is determined. A possible range of the threshold value (th) is as shown in FIG. 3.
When the digital watermark pattern W is embedded in the original picture P, in proportional to the number of changes made to the original picture P, that is, the magnitude of the central position cW·W, the amount of damage to the picture increases. Accordingly, it is a common way to determine scalar value c so that the central position cW·W satisfies desired probabilities PFP and PFN and cW·W is the least. In other words, the relationship represented by the following expression holds. This is specifically represented by the relationship shown in FIG. 4.thFP=thFN=th  (12)
Depending on the application using digital watermarks, extremely small values may be found as the probabilities PFP and PFN. In particular, it is strongly requested that the extremely small value be found. For example, in the case of applying digital watermarks to protection of copyright, if an unauthorized picture is mistakenly determined to be an authorized picture, the determination hardly leads to user complaints, while if an authorized picture is mistakenly determined to be an unauthorized picture, the determination has a large possibility of leading to user complaints. An example, obtained when probabilities PFPand PFN are set to be smaller than those of the example shown in FIG. 4, is shown in FIG. 5.
Values that are limitlessly close to zero are ideal for the probabilities PFP and PFN. Nevertheless, if the threshold value (th) is so determined, the intensity cW·W of digital watermarks to be embedded increases, so that an influence to picture quality of the embedding of the digital watermarks cannot be ignored. The reliability of detection of the digital watermarks and the influence to the picture quality of the digital watermarks have a trade-off relationship.
Various techniques have been proposed which suppresses the influence to the picture quality of the digital watermarks while ensuring the reliability of detection of the digital watermarks. Among them, a widely employed basic technique is to strongly embed digital watermarks in the edge part of an original picture and to weakly embed digital watermarks in the flat part of the picture. This technique uses human visual features in which, when a pixel level is numerically changed, the change is easily noticeable in the flat part, but is hardly noticeable in the edge part. Even in a case in which the intensity of embedding the digital watermarks is set strong and weak in the edge part and the flat part, if a predetermined amount of digital watermark pattern has been embedded as a whole in the picture, the edge part and the flat part have almost identical reliabilities of detection of the digital watermarks.
Multibit Representation of Embedding Information
The process for embedding digital watermark pattern in a picture and the process for detecting the digital watermark pattern have been described. The process for detecting the digital watermark pattern can determine only two states, that is, whether the digital watermark pattern W is embedded or is not embedded. In other words, it can only represent 1-bit information. The following description refers to a method of using a plurality of bits to represent information to be embedded.
Embedding by digital watermarking of multibit information is broadly divided into a type of method that uses a plurality of digital watermark patterns, a type of method that divides a picture into smaller areas, and a composite thereof.
The type of method that uses a plurality of digital watermark patterns includes a method that represents desired information by assigning different meanings to a plurality of digital watermark patterns and exclusively embedding the patterns in a picture, a method that embeds a plurality of digital watermark patterns in a picture so that the patterns are simultaneously superimposed on one another, and uses a combination of the patterns to represent desired information, and a composite of the methods. Embedding of a plurality of digital watermark patterns in an original picture is shown in FIG. 6.
In the method that represents desired information by assigning different meanings to the digital watermark patterns and exclusively embedding the patterns in the picture, when the number of bits for information to be embedded in the picture is b, the required number of digital watermark patterns is n=2b. In the method that embeds a plurality of digital watermark patterns in the picture so that the patterns are simultaneously superimposed on one another, and uses the combination of the patterns to represent the desired information, the required number of digital watermark patterns is n=b. Although the latter needs a small number of digital watermark patterns, in many cases, it may need appropriate measures for coping with deterioration in the picture quality due to the embedding of the digital watermark patterns in the picture. Finally, in the composite of both methods, the required number of digital watermark patterns is b≦n≦2b, and the composite has the features of both methods.
The method that divides a picture into smaller areas is one of a type of embedding by digital watermarking of multibit information. In this method, by assigning different roles to the smaller areas, the picture can be controlled to simultaneously have a plurality of digital watermarks. Various ways for arranging the smaller areas have been proposed. In this embodiment, the arrangement is described below using the example shown in FIG. 7 in which smaller areas are arranged in the form of a grid. In FIG. 7, i and j represent integers that are not negative, respectively.
In the case of dividing the picture into the smaller areas, the divisor is an issue. When information to be embedded in the picture has a size of b bits, division of the picture into b smaller areas is first considered. However, in this method, a problem easily occurs because, in the case of embedding digital watermark patterns in various pictures, embedding of the digital watermark patterns is, in many cases, performed considering visual characteristics of the pictures. For example, when a picture is processed so that digital watermark patterns are strongly embedded in the edge part of the picture and are weakly embedded in the flat part of the picture, if a smaller area corresponding to a certain bit is accidentally a portion of the flat part, a digital watermark pattern embedded in the portion may not be detected. Even in a case in which detection off the digital watermark pattern in one area has failed, if digital watermark patterns in the remaining areas are successfully detected, a combination of all the areas does not have any meaning. When a picture is divided into smaller areas, it is preferable to divide the picture into smaller areas more than b in that stable detection of digital watermark patterns can be performed for various pictures. Even in a case in which the intensity of embedding a digital watermark pattern in one smaller area is extremely weak, if the remaining smaller areas in which identical bit information is embedded sufficiently have the required amount of embedded digital watermark patterns, digital watermarks can be detected as a whole.
FIG. 8 shows an example of division into smaller areas in a case in which embedding information has eight bits. A plurality of smaller areas corresponding to the same bits are assigned in a picture.
Flow of Embedding Digital Watermark Patterns
FIG. 9 shows an example of a process performed in an apparatus that executes embedding of digital watermark patterns in a picture. A digital-watermark-pattern generating unit 1604 generates digital watermark patterns based on embedding information 1602 to be embedded in the picture and a digital-watermark-pattern generating key stored in a digital-watermark-pattern-generating-key storage unit 1603.
The embedding information is information to be embedded as digital watermarks, and includes copy control information, copyright information, and editing information. The digital-watermark-pattern generating key specifically includes picture-division information for use in embedding of digital watermark patterns, and bit-arrangement information, and is the process information required for generating the embedding information as the digital watermark patterns.
The digital-watermark-pattern embedding unit 1605 embeds, in an original picture 1601, the digital watermark patterns generated by the digital-watermark-pattern generating unit 1604. The intensity of embedding the digital watermark patterns in the edge part and flat part of the original picture 1601 is controlled by the digital-watermark-pattern embedding unit 1605. The picture in which the digital watermark patterns are embedded is output as a digital-watermark-embedded picture 1606.
Flow of Digital Watermark Detection
FIG. 10 shows an example of a process performed by an apparatus that executes detection of digital watermark patterns. A digital-watermark-pattern generating unit 1703 generates digital watermark patterns based on a digital-watermark-pattern-generating key stored in a digital-watermark-pattern-generating-key storage unit 1702.
The digital-watermark-pattern-generating key specifically includes picture-division information for use in embedding of the digital watermark patterns, and bit-arrangement information, and is the information required for detection of the digital watermark patterns.
A digital-watermark-pattern correction unit 1704 corrects the digital watermark patterns generated by the digital-watermark-pattern generating unit 1703 by using the picture format of an input picture 1701. The correction performed by the digital-watermark-pattern correction unit 1704 is a process that controls the generated digital watermark patterns to match the size of the input picture 1701 by recognizing the size of the input picture 1701 as a standard definition (SD) picture size having 720 by 480 pixels or as a high definition (HD) picture size having 1920 by 1080 pixels. This process is described next.
A detection unit 1706 detects the digital watermarks of the input picture 1702 by using the digital watermark patterns corrected by the digital-watermark-pattern correction unit 1704. Information detected by the detection unit 1706 is output as detected information 1707.
Picture Format Conversion
HD pictures are provided as picture information for use in television broadcasting by using various types of communication media or storage media. HD picture data represents a picture having a different aspect ratio (the ratio of the width of a picture to the height) from that of a standard (SD) picture of the related art and a different frequency range from that of the standard picture. For example, if a player has no function of processing a high quality picture (hereinafter referred to as an “HD picture”), it performs conversion (downconversion) of the HD picture into the standard picture (hereinafter referred to as the “SD picture”), and displays the SD picture obtained by the conversion. Conversely, it may perform conversion (upconversion) of the SD picture into the HD picture.
Conversion of a picture size, an aspect ratio, etc., is called “picture format conversion”. In the following description, upconversion from the SD picture to the HD picture and downconversion from the HD picture to the SD picture are treated as types of format conversion. When digital watermark patterns are embedded in a picture, and the format of the picture is converted, detection of the embedded digital watermark patterns may be performed by, for example, using correction of the digital watermark patterns by the digital-watermark-pattern correction unit 1704 as described in FIG. 10. However, even if the correction has been performed, it may become difficult to perform detection or an error may occur.
By way of example, FIG. 11A shows that a digital watermark pattern is embedded in an SD picture and the SD picture is upconverted into an HD picture. Although the digital watermark pattern is easily detected from the original SD picture, it is difficult to say that the digital watermark pattern cannot be easily detected from the HD picture. One reason is that picture format conversion is not performed by using integers. For example, if the SD picture has a size of 720 by 480 pixels, and the HD picture has a size of 1920 by 1080 pixels where each picture size represents the product of the number of horizontal pixels and the number of vertical pixels, conversion of the SD picture into the HD picture is performed so that the horizontal direction is 8/3 times and the vertical direction is 9/4 times.
When this conversion using non-integers is performed, the digital watermark pattern embedded in the picture is similarly converted using non-integer magnifications. Since the detection of digital watermarks is executed by multiplying the watermark-embedded picture by the digital watermark pattern, if the embedded digital watermark pattern is slightly deformed, a digital watermark pattern for use in detection of digital watermarks must be similarly deformed. If the digital watermark pattern for detection is appropriately deformed, the detection of digital watermarks may fail.
Conversely, FIG. 11B shows that a digital watermark pattern is embedded in an HD picture, the HD picture is downconverted into an SD picture, and the digital watermark pattern is detected from the SD picture. Similarly to the case shown in FIG. 11A, the case in FIG. 11B shows non-integer conversion. Accordingly, if an appropriately deformed digital watermark pattern is not used for detection, detection of digital watermarks may fail.
How difficult the detection of digital watermarks becomes due to the picture format conversion is described below with reference to an example. Conversion that will be described is shown in FIG. 12. In this conversion, the original image is an SD pixel having a size of 720 by 480 pixels, and a picture generated by the conversion is an HD picture having a size of 1920 by 1080 pixels. However, this conversion is not limited to this combination.
When the SD picture is upconverted into the HD picture, each smaller area of 8 by 8 pixels in the SD picture is converted into each area in the HD picture, as shown in FIG. 13. The smaller area in the SD picture is shown in the lower part of FIG. 13, and when it is converted into the HD picture, that is, it is upconverted, each set of eight vertical pixels in the SD picture is converted into each set of 8 by (1080/480) pixels, and each set of eight horizontal pixels in the SD picture is converted into each set of 8 by (1920/720) pixels. Although no problem occurs if the pixel size is represented by a fraction, pixels represented by fractions, such as ⅓ pixel and ¼ pixel, actually do not exist. In FIG. 13, colors are represented by using the value “1” for white, and using the value “−1” for black. Each rectangular area shown in the upper part of FIG. 13 of the HD picture contains a plurality of pixels, and the substance is as shown in, for example, FIG. 14 or FIGS. 15A and 15B.
FIG. 14 shows an example of an actual form in the HD picture of the smaller area of 8 by 8 pixels in the SD picture. The smaller area of 8 by 8 pixels is converted into an area of the HD picture which has a size of (8 by (1920/720)) by (8 by (1080/480)) pixels, that is, (64/3) by 18 pixels. Actually, pixels represented by fractions, such as ⅓ pixel and ¼ pixel, do not exist. Thus, among pixels constituting the HD picture, pixels positioned in the boundary between black and white are set to be one of black and white. One technique is that, if the white percentage is 50% or greater, the pixels are set be white, and if the black percentage is 50% or greater, the pixels are set to be black. Alternatively, in another technique, the pixels are set to be black if even a small black percentage is set.
As a result, in the case of converting the SD picture into the HD picture, a problem occurs in that the pixel distribution of the SD picture does not accurately correspond to the pixel distribution of the HD picture. In the case of converting the SD picture into the HD picture, 8 vertical pixels correspond to 18 pixels, and each pixel of the SD picture corresponds to 18/8=9/4 pixel. Accordingly, integer pixels in the SD picture, such as one pixel or two pixels, cannot be set as integer pixels in the HD picture, so that fractions are generated. This generates pixels positioned in the boundaries between black and white, as shown in the upper part of FIG. 14. For example, if the white percentage is 50% or greater, the pixels are set to be white, and if the black percentage is 50%, the pixels are set to be black, so that errors are generated.
This similarly applies to horizontal pixels. Each pixel of the SD picture corresponds to (64/3)/8=8/3 pixel, and the integer pixels of the SD picture cannot be set as integer pixels of the HD picture, so that fractions are generated. This generates pixels positioned in the boundaries between black and white, as shown in the upper part of FIG. 14. For example, if the white percentage is 50% or greater, the pixels are set to be white, and if the black percentage is 50%, the pixels are set to be black, so that errors are generated.
FIG. 15 shows another example of the actual form in the HD picture of the smaller area of 8 by 8 pixels in the SD picture. The HD picture shown in the upper part of FIG. 14 is an example obtained by performing processing so that the left end of the HD picture corresponds to that of the SD picture and a ⅔-pixel spare area is generated at the right end of the HD picture. FIG. 15A shows an example of an HD picture obtained by setting the smaller area of 8 by 8 pixels in the center of the HD picture and generating two ⅓-pixel spare pixel areas at the two ends of the HD picture. FIG. 15B shows an example of an HD picture obtained by performing processing so that the right end of the HD picture corresponds to that of the SD picture and a ⅔-pixel spare area is generated at the left end of the HD picture. In both examples, integer pixels cannot be converted into integer pixels by conversion from the SD picture into the HD picture, so that a gap is generated in the percentage of the white area or the percentage of the black area.
As described above, in the process for embedding digital watermarks in the picture, the picture is divided into smaller areas, as shown in FIGS. 7 and 8, and a digital watermark pattern is embedded in each of the smaller areas. Assuming that each of the smaller areas is defined by 8 by 8 pixels of SD picture, smaller areas obtained by dividing the SD picture in units of 8 by 8 pixels, as shown in FIG. 7 or FIG. 8, should correspond to smaller areas obtained by dividing the HD picture in units of (64/3) by 18 pixels.
However, pixels represented by fractions are actually not allowed, as mentioned above. Thus, the smaller areas in the HD picture are set as shown in FIG. 16. In FIG. 16, each hatched portion indicates a portion in which adjacent smaller areas overlap with each other. The hatched portion corresponds to one right-end pixel among the 22 horizontal pixels shown in FIG. 14, one pixel at each end shown in FIG. 15A, or one left-end pixel shown in FIG. 15B.
Since the smaller areas of the HD picture can be set as shown in FIG. 16, it is possible that, when detection from the HD picture of digital watermarks is performed, the detection be performed while ignoring the hatched portions. Compared with the pixel levels of portions other than the hatched portions, the pixel levels of the hatched portions do not include digital watermark patterns. Thus, by using only the pixels of the portions other than the hatched portions, detection of digital watermarks is performed.
Also, it is possible that the HD picture obtained by upconverting the SD picture be processed so that each hatched portion (shown in FIG. 16) in the boundary between two smaller areas is included in either smaller area, as shown in FIG. 17, although it overlaps with the adjacent smaller area. FIG. 17 shows an HD picture obtained by performing processing so that each hatched portion is included in smaller areas that larger occupy the hatched portion. By performing this processing, each smaller area of 8 by 8 pixels in the SD picture can correspond to each smaller area of 21 by 18 pixels or 22 by 18 pixels in the HD picture.
Even in the case of detecting digital watermarks from the above converted pictures by executing the setting of smaller areas as shown in FIGS. 16 and 17, it is noted that the digital watermark pattern must satisfy expression (1).
By way of example, it is assumed that the digital watermark pattern shown in FIG. 14 is embedded in the smaller areas A shown in FIG. 17. Only two types of values, 1 and −1 are used, and an intermediate value is rounded to either value. The digital watermark pattern W in the HD picture is found as shown in FIG. 18. The digital watermark pattern W has a size of 21 by 18 pixels, and 180 ones (white pixels) and 198 negative ones (black pixels). As a result, in the converted HD picture, the left side of expression (1) is not zero, so that expression (1) is not satisfied.
When the digital watermark pattern W does not satisfy expression (1), the result of detection thereof has a high possibility of error. For example, assuming that a digital watermark pattern W which has 8 by 8 pixels and which is composed of positive and negative ones is embedded in a smaller area, 64 is expected as detected value x from the smaller area. The value 64 is obtained by the inner product W·W of digital watermark patterns W (see expressions (7) and (8)). However, if the digital watermark pattern W does not satisfy expression (1), and the sum of the digital watermark pattern W is not zero but one, the product of the one and each pixel level is included as an error.
In other words, in accordance with the above process executed in detection of digital watermarks, that is, expression (7), the inner product x′ of the digital watermark pattern W and the picture M in which the digital watermark pattern W is embedded is calculated. The result of the calculation indicates that the values of P·W included in expression (7) are such relatively larger that they cannot be ignored compared with the inner product W·W. The pixel level is 0 to 255 when it is represented by eight bits. The error having a range of 0 to 255 is sufficient large to deny the meaning of 64 that should be obtained as a detected value of digital watermarks.
In the case of setting the division of the HD picture into smaller areas as shown in FIGS. 16 and 17, even if the digital watermark pattern W shown in FIG. 18 is found with high precision, expression (1) is not satisfied. This is because, at the time the smaller areas are represented as shown in FIGS. 16 and 17, errors due to the rounding of the values of the areas remain.
By adjusting values in the digital watermark pattern W shown in FIG. 18, the digital watermark pattern W can satisfy expression (1). However, accurate detection of digital watermarks is not expected because a digital watermark pattern that originally has errors due to the rounding of the values of smaller areas is adjusted afterward.